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ABSTRACT 



A method for load balancing requests on a network, the 
method including receiving a request from a requestor 
having a requestor network address at a first load balancer 
having a first load balancer network address, the request 
having a source address indicating the requestor network 
address and a destination address indicating the first load 
balancer network address, forwarding the request from the 
first load balancer to a second load balancer at a triangula- 
tion network address, the request source address indicating 
the requestor network address and the destination address 
indicating the triangulation network address, the triangula- 
tion network address being associated with the first load 
balancer network address, and sending a response from the 
second load balancer to the requestor at the requestor 
network address, the response having a source address 
indicating the first load balancer network address associated 
with the triangulation network address and a destination 
address indicating the first requestor network address. 

8 Claims, 9 Drawing Sheets 
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LOAD BALANCING Where redundant server farms are situated in more than 

one geographical location, the geographical location of a 

HELD OF THE INVENTION client may be considered when determining the load bal- 

The present invention relates to computer networks in ^^^er to which the client's requests should be routed, in 

general, and in particular to load balancing cUent requests ' addition to employmg conventional load balancing tech- 

among redundant network servers in different geographical ""'f^^' However, routmg chent requests to the geographi- 

localions ^ ^ nearest server, load balancer, or server farm might not 

necessarily provide the client with the best service if, for 

BACKGROUND OF THE INVENTION example, routing the request to a geographically more 

, , . 10 distant location would otherwise result in reduced latency, 

la computer aetworks, such as the Internet preventing a ^^^^^ ^^ .^^ ^^^^ processing capacity at the 

server from becoming overloaded With requests from clients server 
may be accomplished by providing several servers having 

redundant capabilities and managing the distribution of SUMMARY OF THE INVENTION 

client requests among the servers through a process known ^ . . 

as "load balancine " ^ ^ 15 xhe present mvention seeks to provide novel apparatus 

, . , . ^ , J , , . ,^ and methods for load balancing client requests among 

In one early imp^ementaUon of load balancing a Domain redundant network servers and server farms in different 

Naming System (DNS) server connected to the Internet is ^ical locations which overcome the known disad- 

configured to mainlam several IP addresses for a single ^^^^^ ^^^^ 

domain name, with each address corresponding to one of,n -r^ -u •■j- j ••. e , 

several servers having redundant capabUities. The DNS '^lus provided in accordance with a preferred 

server receives a request for address translation and embodmient of the present mvenuon a method for load 

responds by returning the list ofsetver addresses from which ''^•«"=.'"8 ^"l"*^'^ °° " the method mcluding 

the client chooses one address at random to connect to. fr"™/ '«5"f*'" » '^1"f " 

Alternatively, the DNS server returns a single address cho- „ address at a first load balancer having a first load 

sen either at random or in a round-robin fashion, or actively ' balancer network address, the request having a source 

monitors each of the servers and returns a single address address md.catmg the requestor network address and a 

based on server load and availabUity. destination address mdicatmg the first load balancer network 

, . , . . 4 „ . address, forwarding the request from the first load balancer 

More recently, a device known as a load balancer, such ,^ ^ ^^^^^ balancer at a triangulation network address, 

as the Web Server Director, commercial y avadable from the 30 ^quest source address indicating the requester network 

Apphcant^ssignee, has been used to balance server loads as ^^^^^^ destination address indicating the triangu- 

follows. The load balancer is provided as a gateway o ^^^^^ triangulation network address 

several redundant servers typicjJly situated in a single associated with the first load balancer network 

geographical location and referred to as a server farm or ^ ^^e second load 

"server cluster. DNS serversstore the IP address of the load 35 balancer to the requestor at the requester network address, 
balancer rather than the addresses of the servers to which the 
load balancer is connected. Ilie load balancer's address is 
referred to as a "virtual IP address" in that it masks the 
addresses of the servers to which it is connected. Client 

requests are addressed to the virtual IP address of the load 40r^,. . rjuj-.r.u 

balancer which then sends the request to a server based on P"^''" accordance with a preferred embodmient of the 

server load and avaUability or using other known tech- P«*°' J"^"*'"" °?^'bod includes maintaining the asso- 

ciation between the triangulation network address and the 

, * , , . ... ... J first load balancer network address at either of the load 

Just as redundant servers in combination with a load balancers 

balancer may be used to prevent server overload, redundant 45 „ ^ * , . ' , . , ^ ^ , j. r 

server farms may be used to reroute client requests received , ^tiU further m accorfance wth a preferred embodiment of 

at a first load balancer/server farm to a second load balancer/ "'^ P^*^"' '^e method mcludcs mamtammg the 

server farm where none of the servers in the first server farm association between the triangulation network address and 

are available to tend to the request. One rerouting method '^e first load balancer network address at the second oad 

currently being used involves sending an HTTP redirect 50 bakncer, and communicating the association to the first load 

message from the first load balancer/server farm to the client ^ ancer. 

instructing the client to reroute the request to the second load Additionally in accordance with a preferred embodiment 
balancer/server farm indicated in the redirect message. This of the present invention the method mcludes directmg the 
method of load balancing is disadvantageous in that it can request from the second load balancer to a server m corn- 
only be employed in response to HTTP requests, and not for 55 mumcation with the second load balancer, composing the 
other types of requests such as FTP requests. Another response at the server, and providmg the response to the 
rerouting method involves configuring the first load balancer second load balancer. 

to act as a DNS server. Upon receiving a DNS request, the There is also provided in accordance with a preferred 

first load balancer simply returns the virtual IP address of the embodiment of the present invention a method for load 

second load balancer. This method of load balancing is 60 balancing requests on a network, the method including 

disadvantageous in thai it can only be employed in response determining the network proximity of a requester with 

to DNS requests where there is no guarantee that the request respect to each of at least two load balancers, designating a 

will come to the first load balancer since the request does not closest one of the load balancers by ranking the load 

come directly from the client, and where subsequent balancers by network proximity, and directing requests from 

requests to intermediate DNS servers may result in a pre- 65 the requestor to the closest load balancer, 

viously cached response being relumed with a virtual IP Further in accordance with a preferred embodiment of the 

address of a load balancer that is no longer available. present invention the method includes directing requests 



the response having a source address indicating the first load 
balancer network address associated with the triangulation 
network address and a destination address indicating the first 
requestor network address. 
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from any source having a subnet that is the same as the 
subnet of the requestor to the closest load balancer 

Still further in accordance with a preferred embodiment of 
the present invention the method includes monitoring the 
current load of each of the load balancers, and performing 
the directing step the current load of the closest load 
balancer is less than the current load of every other of the 
load balancers. 

Additionally in accordance with a preferred embodiment 
of the present invention the determining step includes peri- 
odically determining. 

Moreover in accordance with a preferred embodiment of 
the present invention the determining step includes deter- 
mining at at least one fixed time. 

Further in accordance with a preferred embodiment of the 
present invention the determining step includes polling the 
requestor to yield atJeast two attributes selected from the 
group consisting of: latency, relative TTL, and number of 
hops to requestor. 

Still further in accordance with a preferred embodiment of 
the present invention the determining step includes polling 
the requestor using at least two polling methods selected 
from the group consisting of: pinging, sending a TCP ACK 
message to the requestor's source address and port, sending 
a TCP ACK message to the requestor's source address and 
port 80, and sending a UDP request to a sufiSciently high port 
number as to elicit an "ICMP port unreachable'' reply. 

Additionally in accordance with a preferred embodiment 
of the present invention the designating step includes des- 
ignating a closest one of the load balancers by ranking the 
load balancers by network proximity and either of current 
load and available capacity. 

There is also provided in accordance with a preferred 
embodiment of the present invention a method for deter- 
mining network proximity, the method including sending 
from each of at least two servers a UDP request having a 
starting TTL value to a client at a sufficiently high port 
number as to elicit an "ICMP port unreachable" reply 
message lo at least one determining one of the servers 
indicating the UDP request's TTL value on arrival at the 
client, determining a number of hops from each of the 
servers to the client by subtracting the starting TTL value 
from the TTL value on arrival for each of the servers, and 
determining which of the servers has fewer hops of the 
client, and designating the server having fewer hops as being 
closer to the client than the other of the servers. 

There is additionally provided in accordance with a 
preferred embodiment of the present invention a network 
load balancing system including a network, a first load 
balancer connected to the network and having a first load 
balancer network address, a second load balancer connected 
to the network and having a triangulation network address, 
the triangulation network address being associated with the 
first load balancer network address, and a requestor con- 
nected to the network and having a requester network 
address, where the requestor is operative to send a request 
via the network to the first load balancer, the request having 
a source address indicating the requestor network address 
and a destination address indicating the first load balancer 
network address, the first load balancer is operative to 
forward the request to the second load balancer at the 
triangulation network address, the request source address 
indicating the requestor network address and the destination 
address indicating the triangulation network address, and the 
second load balancer is operative to send a response lo the 
requester at the requester network address, the response 
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having a source address indicating the first load balancer 
network address associated with the triangulation network 
address and a destination address indicating the first 
requestor network address. 
5 Further in accordance with a preferred embodiment of the 
present invention either of the load balancers is operative to 
maintain a table of the association between the triangulation 
network address and the first load balancer network address. 

Still further in accordance with a preferred embodiment of 
the present invention the second load balancer is operative 
to maintain a table of the association between the triangu- 
lation network address and the first load balancer network 
address and communicate the association to the first load 
balancer. 

Additionally in accordance with a preferred embodiment 
of the present invention the system further includes a server 
in communication with the second load balancer, where the 
second load balancer is operative to direct the request from 
the second load balancer to the server, and the server is 

20 

Operative to compose the response and provide the response 
to the second load balancer. 

There is also provided in accordance with a preferred 
embodiment of the present invention a network load bal- 

25 ancing system including a network, at least two load bal- 
ancers connected to the network, and a requester connected 
to the network, where each of the at least two load balancers 
is operative to determine the network proximity of the 
requester, and at least one of the load balancers is operative 
to designate a closest one of the load balancers by ranking 
the load balancers by network proximity and direct requests 
fi-om either of the requestor and a subnet of the requestor to 
the closest load balancer 
Further in accordance with a preferred embodiment of the 

35 present invention the load balancers are operative to poll the 
requestor to yield at least two attributes selected from the 
group consisting of: latency, relative TTL, and number of 
hops to requester. 
Still further in accordance with a preferred embodiment of 

40 the present invention the load balancers are operative to poll 
the requester using at least two polling methods selected 
fi-om the group consisting of: pinging, sending a TCP ACK 
message to the requestor's source address and port, sending 
a TCP ACK message to the requestor's source address and 

45 port 80, and sending a UDP request to a sufficiently high port 
number as to elicit an "ICMP port unreachable" reply. 

Additionally in accordance with a preferred embodiment 
of the present invention at least one of the load balancers is 
operative to designate the closest one of the load balancers 

50 by ranking the load balancers by network proximity and 
either of current load and available capacity. 

It is noted that throughout the specification and claims the 
term "network proximity" refers to the quality of the rela- 
tionship between a chent and a first server or server farm as 

55 compared with the relationship between the chent and a 
second server or server farm when collectively considering 
multiple measurable factors such as latency, hops, and server 
processing capacity. 

BRIEF DESCRIPTION OF THE DRAWINGS 

ou 

The present invention will be understood and appreciated 
from the following detailed description, taken in conjunction 
with the drawings in which: 

FIGS. lA-lC, taken together, are simplified pictorial flow 
65 illustrations of a triangulation load balancing system con- 
structed and operative in accordance with a preferred 
embodiment of the present invention; and 
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FIGS. 2A-2F, taken together, are simplified pictorial flow 200.100.1.1, checks triangulation mapping table 32 and 

illustrations of a network proximity load balancing system finds that virtual IP address 200.100.1.1 has been designated 

constructed and operative in accordance with another pre- for LBl's use. LB2 therefore uses the virtual IP address 

ferred embodiment of the present invention. 100.100.1.0 of LBl as per triangulation mapping table 32 as 

5 the source IP address of an outgoing response 40 that LB2 

DETAILED DESCRIPTION OF PREFERRED sends to client 26 after the request has been serviced by one 

EMBODIMENTS Ihe servers in server farm 12 selected by LB2, It is 

appreciated that response 40 must appear to client 26 to 

Reference is now made to FIGS. lA-lC which, taken come from LBl, otherwise client 26 will simply ignore 

together, are simplified pictorial flow illustrations of a response 40 as an unsolicited packet. Client 26 may continue 

triangulation load balancing system constructed and opera- to send requests to LBl which LBl then forwards requests 

tive in accordance with a preferred embodiment of the to LB2 at the designated triangulation address. LB2 directs 

present invention. Two server farms, generally designated requests to an available server and sends responses to client 

10 and 12 respectively, are shown connected to a network 26 indicating LBl as the source IP address. 

14, such as the Internet, although it is appreciated that more Reference is now made to FIGS. 2A-2F which, taken 

than two server farms may be provided. Server farms 10 and together, are simplified pictorial flow illustrations of a 

12 typically comprise a load balancer 16 and 18 respectively, network proximity load balancing system constructed and 

which may be a dedicated load balancer or a server or router operative in accordance with another preferred embodiment 

configured to operate as a load balancer, with each of the of present invention. The configuration of the system of 

load balancers being connected to one or more servers 20. pjcs. 2A-2F is substantially similar to FIGS. lA-lC except 

Load balancers 16 and 18 are alternatively referred to herein as otherwise described hereinbelow. For illustration 

as LBl and LB2 . respectively. LBl and LB2 typicaUy purposes, a third server farm, generaUy designated 50, is 

maintain a server status table 22 and 24 respectively, indi- shown connected to network 14, although it is appreciated 

eating the current load, configuration, availability, and other that two or more server farms may be provided. Server farm 

server information as is common to load balancers. LBl and 50 typically comprises a load balancer 52, which may be a 

LB2 also typically periodicaUy receive and maintain each dedicated load balancer or a server or router configured to 

other's overaU status and load statistics such that LBl and operate as a load balancer, with load balancer 52 being 

LB2 can know each other's availabiHty. connected to two or more servers 20. Load balancer 52 is 

Typical operation of the triangulation load balancing alternatively referred to herein as LB3. 
system of HGS. lA-lC is now described by way of 3Q Typical operation of the network proximity load balanc- 

example. As is shown more particularly with reference to ing system of FIGS, 2A-2F is now described by way of 

FIG. lA, a client 26, such as any known computer terminal example. As is shown more particularly with reference to 

configured for communication via network 14, is shown FIG. 2A, client 26 is shown sending request 28, such as an 

sending a request 28, such as an FTP or HTTP request, to FTP or HTTP request, to LBl whose virtual IP address is 
LBl whose virtual IP address is 100.100.1.0. In accordance 33 100.100.1.0. LBl preferably maintains a proximity table 54 

with network transmission protocols, request 28 indicates indicating subnets and the best server farm site or sites to 

the source IP address of the requester, being the IP address which requests from a particular subnet should be routed. 

197.1.33.5of client 26, and the destination IP address, being Determining the "best" site is described in greater detail 

the virtual IP address 100.100.1.0 of LBl. LB2 preferably hereinbelow. 

periodically sends a status report 30 to LBl, the virtual IP Upon receiving a request, LBl may decide to service the 

address lOO.lOO.l.OofLBlbeingknown in advance to LB2. ^^ciyiesi or not based on normal load balancing consider- 

Status report 30 typically indicates the avaUabifity of server ^^i^j^^. In any case, LBl may check proximity table 54 for 

farm 12 and provides load statistics, which LBl maintains. ^n entry indicating the subnet corresponding to the subnet of 

LB2 is preferably capable of having multiple virtual IP the source IP address of the incoming request. As is shown 
addresses as is well known. It is a particular feature of the 45 more particularly with reference to FIG. 2B, if no corre- 

present invention for LB2 to designate a currently unused sponding entry is found in proximity table 54, LBl may send 

virtual IP address, such as 200.100.1.1, for LBTs use and a proximity request 56 to LB2, and LB3, whose virtual IP 

store the mapping between the IP address of LBl and the addresses are known in advance to LBl. Proximity request 

designated IP address in a triangulation mapping table 32, as 56 indicates the IP address of client 26. 
is shown more particularly with reference to FIG. IB. The 50 A "network proximity" may be detennined for a requester 

designated address is referred to herein as the triangulation such as client 26 with respect to each load balancer/server 

address and may be preconfigu red with LBl or periodicaUy farm by measuring and coflectively considering various 

provided to LBl from LB2. LBl preferably maintains in a attributes of the relationship such as latency, hops between 

client mapping table 36 a mapping of the IP address client 26 and each server farm, and the processing capacity 
197.1.33.5 of client 26 and the triangulation address 55 and quality of each server farm site. To determine compara- 

200.100.1.1 of LB2 to which client 26's requests may be tive network proximity, LBl, LB2, and LB3 preferably each 

redirected. send a polling request 58 to chent 26 using known polling 

As shown in the example of FIG. lA, server status table mechanisms. While known polling mechanisms included 

22 of LBl indicates that no servers in server farm 10 are pinging client 26, sending a TCP ACK message to client 26 
available to service client 26 's request, but indicates that 60 may be used where pinging would otherwise fail due to an 

server farm 12 is available. Having decided that client 26' s intervening firewall or NAT device filtering out a polling 

request should be forwarded to LB2, in FIG. IC LBl message. A TCP ACK may be sent to the client's source IP 

substitutes the destination IP address of request 28 with the address and port. If the client's request was via a UDP 

virtual IP address 200. 100.1 .1 of LB2 which is now mapped connection, a TCP ACK to the client's source IP address and 
to the IP address of client 26 as per client mapping table 36 65 port 80 may be used. One or both TCP ACK messages 

and sends an address- modified client request 38 to LB2. should bypass any intervening NAT or firewall and cause 

LB 2, upon receiving request 38 at its virtual IP address client 26 to send a TCP RST message, which may be used 
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to determine both latency and TTL. While TTL does not 
necessarily indicate the number of hops from the client to the 
load balancer, comparing TTL values from LBl, LB2, and 
LB3 should indicate whether it took relatively more or less 
hops. 5 

Another polling method involves sending a UDP request* 
to a relatively high port number at the client, such as 2090. 
This request would typically be answered with an "I CMP 
port unreachable" reply which would indicate the TTL value 
of the UDP request on arrival at the client. Since the starting lo 
TTL value of each outgoing UDP request is known, the 
actual number of hops to the client may be determined by 
subtracting the TTL value on arrival at the client from the 
starting TTL value. A combination of pinging, TCP ACK, 
UDP, and other polling techniques may be used since any ^5 
one polling request might fail. 

Qient 26 is shown in FIG. 2D sending a polling response 
60 to the various polling requests. The responses may be 
used to determine the latency of the transmission, as well as 
the TTL value. LB2 and LB3 then send polling results 62 to 
LBl, as shown in FIG. 2E. The polling results may then be 
compared, and LBl, LB2, and LB3 ranked, such as by 
weighting each attribute and determining a total weighted 
value for each server farm. Polling results may be consid- 
ered together with server farm capacity and availability, such 
as may be requested and provided using known load bal- 
ancing reporting techniques or as described hereinabove 
with reference to FIGS. lA and IB, to determine the server 
farm site that is "closest" to client 26 and, by extension, the 
Ghent's subnet, which, in the example shown, is determined "^^ 
to be LB2. For example, the closest site may be that which 
has the lowest total weighted value for all polhng, load, and 
capacity results. LBl may then store the closest site to the 
client/subnet in proximity table 54. 

As was described above, a load balancer that receives a 
request from a cHent may check proximity table 54 for an 
entry indicating the subnet corresponding to the subnet of 
the source IP address of the incoming request. Thus, if a 
corresponding entry is found in proximity table 54, the 
request is simply routed to the location having the best 
network proximity. Although the location having the best 
network proximity to a particular subnet may have already 
been determined, the load balancer may nevertheless decide 
to forward an incoming request to a location that does not 
have the best network proximity should a load report 
received from the best location indicate that the location is 
too busy to receive requests. In addition, the best network 
proximity to a particular subnet may be periodically 
redetermined, such as at fixed times or after a predetermined 
amount of time has elapsed from the time the last determi- 
nation was made. 

As is shown more particularly with reference to FIG. 2F, 
once the closest site for chent 26 has been determined, cUent 
26 may be redirected to the closest site using various 
methods. If a DNS request is received from client 26, LBl 
may respond with LB2's address. If an HTTP request is 
received from client 26, HTTP redirection may be used. 
Alternatively, regardless of the type of request received from 
client 26, triangulation as described hereinabove with ref- 
erence to FIGS. lA-lC may be used. 

It is appreciated that elements of the present invention 
described hereinabove may be implemented in hardware, 
software, or any suitable combination thereof using conven- 
tional techniques, 65 

It is appreciated that the steps described with reference to 
FIGS. lA-lC and 2A-2F need not necessarily be performed 
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in the order shown unless otherwise indicated, and that in 
fact different implementations of the steps may be employed 
to yield similar overall resuhs. 

It is appreciated that various features of the invention 
which are, for cliarity, described in the contexts of separate 
embodiments may also be provided in combination in a 
single embodiment. Conversely, various features of the 
invention which are, for brevity, described in the context of 
a single embodiment may also be provided separately or in 
any suitable subcombination. . 

It will be appreciated by persons skilled in the art that the 
present invention is not limited to what has been particularly 
shown and described hereinabove. Rather, the scope of the 
present invention is defined only by the claims that follow: 

What is claimed is: 

1. A method for load balancing requests on a network, the 
method comprising: 

receiving a request from a requester having a requestor 
network address at a first load balancer having a first 
load balancer network address, said request having a 
source address indicating said requester network 
address and a destination address indicating said first 
load balancer network address; 

forwarding said request from said first load balancer to a 
second load balancer at a triangulation network 
address, said request source address indicating said 
requestor network address and said destination address 
indicating said triangulation network address, said tri- 
angulation network address being associated with said 
first load balancer network address; and 

sending a response from said second load balancer to said 
requester at said requestor network address, said 
response having a source address indicating said first 
load balancer network address associated with said 
triangulation network address and a destination address 
indicating said first requester network address. 

2. A method according to claim 1 and further comprising: 
maintaining said association between said triangulation 

network address and said first load balancer network 
address at either of said load balancers. 

3. A method according to claim 1 and further comprising: 
maintaining said association between said triangulation 

network address and said first load balancer network 
address at said second load balancer; and 
communicating said association to said first load balancer. 

4. A method according to claim 1 and further comprising: 
directing said request from said second load balancer to a 

server in communication with said secorid load bal- 
ancer; 

composing said response at said server; and 
providing said response to said second load balancer. 

5. A network load balancing system comprising: 
a network; 

a first load balancer connected to said network and having 
a first load balancer network address; 

a second load balancer connected to said network and 
having a triangulation network address, said triangula- 
tion network address being associated with said first 
load balancer network address; and 

a requestor connected to said network and having a 
requester network address, 

wherein said requestor is operative lo send a request via 
said network to said first load balancer, said request 
having a source address indicating said requester net- 
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work address and a destination address indicating said 
first load balancer network address, wherein said first 
load balancer is operative to forward said request to 
said second load balancer at said triangulation network 
address, said request source address indicating said 5 
requestor network address and said destination address 
indicating said triangulation network address, and 
wherein said second load balancer is operative to send 
a response to said requester at said requestor network 
address, said response having a source address indicat- lO 
ing said first load balancer network address associated 
with said triangulation network address and a destina- 
tion address indicating said first requester network 
address. 

6. A system according to claim 5 wherein either of said as 
load balancers is operative to maintain a table of said 
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association between said triangulation network address and 
said first load balancer network address. 

7. A system according to claim 5 wherein said second load 
balancer is operative to maintain a table of said association 
between said triangulation network address and said first 
load balancer network address and communicate said asso- 
ciation to said first load balancer. 

8. A system according to claim 5 and further comprising 
a server in communication with said second load balancer, 
wherein said second load balancer is operative to direct said 
request from said second load balancer to said server, and 
wherein said server is operative to compose said response 
and provide said response to said second load balancer. 

***** 



04/12/2004, EAST Version: 1.4.1 



